{
    "cells": [
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<a href=\"http://cocl.us/pytorch_link_top\">\n    <img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/Pytochtop.png\" width=\"750\" alt=\"IBM Product \" />\n</a> "
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/cc-logo-square.png\" width=\"200\" alt=\"cognitiveclass.ai logo\" />"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h1><h1>Pre-trained-Models with PyTorch </h1>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "In this lab, you will use pre-trained models to classify between the negative and positive samples; you will be provided with the dataset object. The particular pre-trained model will be resnet18; you will have three questions: \n<ul>\n<li>change the output layer</li>\n<li> train the model</li> \n<li>  identify  several  misclassified samples</li> \n </ul>\nYou will take several screenshots of your work and share your notebook. "
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2>Table of Contents</h2>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n\n\n<ul>\n    <li><a href=\"#download_data\"> Download Data</a></li>\n    <li><a href=\"#auxiliary\"> Imports and Auxiliary Functions </a></li>\n    <li><a href=\"#data_class\"> Dataset Class</a></li>\n    <li><a href=\"#Question_1\">Question 1</a></li>\n    <li><a href=\"#Question_2\">Question 2</a></li>\n    <li><a href=\"#Question_3\">Question 3</a></li>\n</ul>\n<p>Estimated Time Needed: <strong>120 min</strong></p>\n </div>\n<hr>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2 id=\"download_data\">Download Data</h2>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Download the dataset and unzip the files in your data directory, unlike the other labs, all the data will be deleted after you close  the lab, this may take some time:"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "!wget https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Positive_tensors.zip ",
            "execution_count": 8,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "--2020-04-17 14:08:20--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Positive_tensors.zip\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 2598656062 (2.4G) [application/zip]\nSaving to: \u2018Positive_tensors.zip\u2019\n\n100%[====================================>] 2,598,656,062 45.4MB/s   in 57s    \n\n2020-04-17 14:09:18 (43.2 MB/s) - \u2018Positive_tensors.zip\u2019 saved [2598656062/2598656062]\n\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "!unzip -q Positive_tensors.zip ",
            "execution_count": 9,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "! wget https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Negative_tensors.zip\n!unzip -q Negative_tensors.zip",
            "execution_count": 10,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "--2020-04-17 14:12:21--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Negative_tensors.zip\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 2111408108 (2.0G) [application/zip]\nSaving to: \u2018Negative_tensors.zip\u2019\n\n100%[====================================>] 2,111,408,108 44.6MB/s   in 45s    \n\n2020-04-17 14:13:06 (44.6 MB/s) - \u2018Negative_tensors.zip\u2019 saved [2111408108/2111408108]\n\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "We will install torchvision:"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "!pip install torchvision",
            "execution_count": 3,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "Collecting torchvision\n\u001b[?25l  Downloading https://files.pythonhosted.org/packages/7e/90/6141bf41f5655c78e24f40f710fdd4f8a8aff6c8b7c6f0328240f649bdbe/torchvision-0.5.0-cp36-cp36m-manylinux1_x86_64.whl (4.0MB)\n\u001b[K     |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 4.0MB 18.6MB/s eta 0:00:01\n\u001b[?25hRequirement already satisfied: six in /opt/conda/envs/Python36/lib/python3.6/site-packages (from torchvision) (1.12.0)\nCollecting torch==1.4.0 (from torchvision)\n\u001b[?25l  Downloading https://files.pythonhosted.org/packages/24/19/4804aea17cd136f1705a5e98a00618cb8f6ccc375ad8bfa437408e09d058/torch-1.4.0-cp36-cp36m-manylinux1_x86_64.whl (753.4MB)\n\u001b[K     |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 753.4MB 50kB/s s eta 0:00:01MB/s eta 0:00:13\ufffd\ufffd\u2588\u2588\u258e                        | 170.3MB 47.6MB/s eta 0:00:134MB 47.6MB/s eta 0:00:137MB 52.0MB/s eta 0:00:11\ufffd\ufffd\u2588\u2588\u2588\u2588\u2588                     | 260.8MB 51.5MB/s eta 0:00:10\ufffd\u2588\u2588\u258e                 | 335.8MB 52.3MB/s eta 0:00:08MB/s eta 0:00:03\n\u001b[?25hRequirement already satisfied: numpy in /opt/conda/envs/Python36/lib/python3.6/site-packages (from torchvision) (1.15.4)\nRequirement already satisfied: pillow>=4.1.1 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from torchvision) (5.4.1)\nInstalling collected packages: torch, torchvision\nSuccessfully installed torch-1.4.0 torchvision-0.5.0\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2 id=\"auxiliary\">Imports and Auxiliary Functions</h2>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "The following are the libraries we are going to use for this lab. The <code>torch.manual_seed()</code> is for forcing the random function to give the same number every time we try to recompile it."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# These are the libraries will be used for this lab.\nimport torchvision.models as models\nfrom PIL import Image\nimport pandas\nfrom torchvision import transforms\n\nimport torch.nn as nn\nimport time\nimport torch \nimport matplotlib.pylab as plt\nimport numpy as np\nfrom torch.utils.data import Dataset, DataLoader\nimport h5py\nimport os\nimport glob\ntorch.manual_seed(0)",
            "execution_count": 4,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 4,
                    "data": {
                        "text/plain": "<torch._C.Generator at 0x7fd33c022fd0>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "from matplotlib.pyplot import imshow\nimport matplotlib.pylab as plt\nfrom PIL import Image\nimport pandas as pd\nimport os",
            "execution_count": 5,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<!--Empty Space for separating topics-->"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2 id=\"data_class\">Dataset Class</h2>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": " This dataset class is essentially the same dataset you build in the previous section, but to speed things up, we are going to use tensors instead of jpeg images. Therefor for each iteration, you will skip the reshape step, conversion step to tensors and normalization step."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# Create your own dataset object\n\nclass Dataset(Dataset):\n\n    # Constructor\n    def __init__(self,transform=None,train=True):\n        directory=\"/home/dsxuser/work\"\n        positive=\"Positive_tensors\"\n        negative='Negative_tensors'\n\n        positive_file_path=os.path.join(directory,positive)\n        negative_file_path=os.path.join(directory,negative)\n        positive_files=[os.path.join(positive_file_path,file) for file in os.listdir(positive_file_path) if file.endswith(\".pt\")]\n        negative_files=[os.path.join(negative_file_path,file) for file in os.listdir(negative_file_path) if file.endswith(\".pt\")]\n        number_of_samples=len(positive_files)+len(negative_files)\n        self.all_files=[None]*number_of_samples\n        self.all_files[::2]=positive_files\n        self.all_files[1::2]=negative_files \n        # The transform is goint to be used on image\n        self.transform = transform\n        #torch.LongTensor\n        self.Y=torch.zeros([number_of_samples]).type(torch.LongTensor)\n        self.Y[::2]=1\n        self.Y[1::2]=0\n        \n        if train:\n            self.all_files=self.all_files[0:30000]\n            self.Y=self.Y[0:30000]\n            self.len=len(self.all_files)\n        else:\n            self.all_files=self.all_files[30000:]\n            self.Y=self.Y[30000:]\n            self.len=len(self.all_files)     \n       \n    # Get the length\n    def __len__(self):\n        return self.len\n    \n    # Getter\n    def __getitem__(self, idx):\n               \n        image=torch.load(self.all_files[idx])\n        y=self.Y[idx]\n                  \n        # If there is any transform method, apply it onto the image\n        if self.transform:\n            image = self.transform(image)\n\n        return image, y\n    \nprint(\"done\")",
            "execution_count": 6,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "done\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "We create two dataset objects, one for the training data and one for the validation data."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "train_dataset = Dataset(train=True)\nvalidation_dataset = Dataset(train=False)\nprint(\"done\")",
            "execution_count": 13,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "done\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2 id=\"Question_1\">Question 1</h2>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<b>Prepare a pre-trained resnet18 model :</b>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<b>Step 1</b>: Load the pre-trained model <code>resnet18</code> Set the parameter <code>pretrained</code> to true:"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# Step 1: Load the pre-trained model resnet18\nmodel = models.resnet18(pretrained=True)",
            "execution_count": 12,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "Downloading: \"https://download.pytorch.org/models/resnet18-5c106cde.pth\" to /home/dsxuser/.cache/torch/checkpoints/resnet18-5c106cde.pth\n",
                    "name": "stderr"
                },
                {
                    "output_type": "display_data",
                    "data": {
                        "text/plain": "HBox(children=(IntProgress(value=0, max=46827520), HTML(value='')))",
                        "application/vnd.jupyter.widget-view+json": {
                            "version_major": 2,
                            "version_minor": 0,
                            "model_id": "eb25db163fa34f78b005aac9dc517018"
                        }
                    },
                    "metadata": {}
                },
                {
                    "output_type": "stream",
                    "text": "\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<b>Step 2</b>: Set the attribute <code>requires_grad</code> to <code>False</code>. As a result, the parameters will not be affected by training."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# Step 2: Set the parameter cannot be trained for the pre-trained model\nfor param in model.parameters():\n    param.requires_grad = False",
            "execution_count": 15,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<code>resnet18</code> is used to classify 1000 different objects; as a result, the last layer has 1000 outputs.  The 512 inputs come from the fact that the previously hidden layer has 512 outputs. "
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<b>Step 3</b>: Replace the output layer <code>model.fc</code> of the neural network with a <code>nn.Linear</code> object, to classify 2 different classes. For the parameters <code>in_features </code> remember the last hidden layer has 512 neurons."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "model.fc = nn.Linear(512,2)",
            "execution_count": 28,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Print out the model in order to show whether you get the correct answer.<br> <b>(Your peer reviewer is going to mark based on what you print here.)</b>"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "print(model)",
            "execution_count": 29,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "ResNet(\n  (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n  (relu): ReLU(inplace=True)\n  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n  (layer1): Sequential(\n    (0): BasicBlock(\n      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      (relu): ReLU(inplace=True)\n      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n    )\n    (1): BasicBlock(\n      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      (relu): ReLU(inplace=True)\n      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n    )\n  )\n  (layer2): Sequential(\n    (0): BasicBlock(\n      (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      (relu): ReLU(inplace=True)\n      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      (downsample): Sequential(\n        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)\n        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      )\n    )\n    (1): BasicBlock(\n      (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      (relu): ReLU(inplace=True)\n      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n    )\n  )\n  (layer3): Sequential(\n    (0): BasicBlock(\n      (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      (relu): ReLU(inplace=True)\n      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      (downsample): Sequential(\n        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)\n        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      )\n    )\n    (1): BasicBlock(\n      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      (relu): ReLU(inplace=True)\n      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n    )\n  )\n  (layer4): Sequential(\n    (0): BasicBlock(\n      (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      (relu): ReLU(inplace=True)\n      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      (downsample): Sequential(\n        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      )\n    )\n    (1): BasicBlock(\n      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      (relu): ReLU(inplace=True)\n      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n    )\n  )\n  (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))\n  (fc): Linear(in_features=512, out_features=2, bias=True)\n)\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2 id=\"Question_2\">Question 2: Train the Model</h2>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "In this question you will train your, model:"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<b>Step 1</b>: Create a cross entropy criterion function "
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# Step 1: Create the loss function\n\n# Type your code here\ncriterion = nn.CrossEntropyLoss()",
            "execution_count": 30,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<b>Step 2</b>: Create a training loader and validation loader object, the batch size should have 100 samples each."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "train_loader = torch.utils.data.DataLoader(dataset = train_dataset,batch_size = 15)\nvalidation_loader = torch.utils.data.DataLoader(dataset = validation_dataset,batch_size = 10)",
            "execution_count": 31,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<b>Step 3</b>: Use the following optimizer to minimize the loss "
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "optimizer = torch.optim.Adam([parameters  for parameters in model.parameters() if parameters.requires_grad],lr=0.001)",
            "execution_count": 32,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<!--Empty Space for separating topics-->"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "**Complete the following code to calculate  the accuracy on the validation data for one epoch; this should take about 45 minutes. Make sure you calculate the accuracy on the validation data.**"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "n_epochs=1\nloss_list=[]\naccuracy_list=[]\ncorrect=0\nN_test=len(validation_dataset)\nN_train=len(train_dataset)\nstart_time = time.time()\n#n_epochs\n\nLoss=0\nstart_time = time.time()\nfor epoch in range(n_epochs):\n    for x, y in train_loader:\n\n        model.train() \n        #clear gradient \n        optimizer.zero_grad()\n        #make a prediction \n        z = model(x)\n        # calculate loss \n        loss = criterion(z,y)\n        # calculate gradients of parameters \n        loss.backward()\n        # update parameters \n        optimizer.step()\n        loss_list.append(loss.data)\n    correct=0\n    for x_test, y_test in validation_loader:\n        # set model to eval \n        model.eval()\n        #make a prediction \n        z = model(x_test)\n        #find max \n        _, yhat = torch.max(z.data, 1)\n\n        \n        correct += (yhat==y_test).sum().item()\n        \n   \n    accuracy=correct/N_test\n\n",
            "execution_count": 33,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<b>Print out the Accuracy and plot the loss stored in the list <code>loss_list</code> for every iteration and take a screen shot.</b>"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "accuracy",
            "execution_count": 34,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 34,
                    "data": {
                        "text/plain": "0.9943"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "plt.plot(loss_list)\nplt.xlabel(\"iteration\")\nplt.ylabel(\"loss\")\nplt.show()\n",
            "execution_count": 35,
            "outputs": [
                {
                    "output_type": "display_data",
                    "data": {
                        "text/plain": "<Figure size 432x288 with 1 Axes>",
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XmYFNW5BvD3m4UBZF/cWBxUENFEVERwyWoQjFdvNHFJzDXRRL03JjFek2BwxdxgYpS4K4nESIzigkoEBARl32bYd8ZhBgaG2Zm9Z+tz/+jqnuqequ7qpbq6qff3PDx011RXfV3dfb46S50SpRSIiIgAIMPpAIiIKHUwKRARUQCTAhERBTApEBFRAJMCEREFMCkQEVEAkwIREQUwKRARUQCTAhERBWQ5HUC0Bg0apHJzc50Og4goreTn51cqpQZHWi/tkkJubi7y8vKcDoOIKK2ISLGV9dh8REREAUwKREQUwKRAREQBTApERBTApEBERAFMCkREFMCkQEREAa5JCpuKqvH0kn1o6/A6HQoRUcpyTVLYXFyD55cXMCkQEYXhmqSQmSEAgHavcjgSIqLU5bqk0NHBpEBEZMY1SSHLnxQUkwIRkRnXJIXMDN9b7WDzERGRKRclBd//7FMgIjLnoqTge6teJgUiIlOuSQpZHH1ERBSRa5JChr+j2cvrFIiIzLgmKQRGHzEnEBGZck1S8F+nwCuaiYjM2ZoURGSyiOwTkQIRmWrw9+Ei8pmIbBGR7SJyjV2x9M7x3Y66ztNm1y6IiNKebUlBRDIBvAhgCoAxAG4VkTEhqz0E4B2l1IUAbgHwkl3xDOqdAwCoami1axdERGnPzprCeAAFSqlCpVQrgLcBXB+yjgLQR3vcF8BRu4LJzuTFa0REkdiZFIYAOKx7XqIt03sMwG0iUgJgIYCfG21IRO4SkTwRyauoqIgpGK1LAV5Oc0FEZMrOpCAGy0JL5FsBvK6UGgrgGgBzRKRLTEqpWUqpcUqpcYMHD44pmAzxjz5iUiAiMmNnUigBMEz3fCi6Ng/dCeAdAFBKrQPQHcAgO4LxX6fAigIRkTk7k8ImACNFZISIdIOvI3l+yDqHAHwTAETkXPiSQmztQxGw+YiIKDLbkoJSqh3AvQAWA9gD3yijXSIyXUSu01b7XwA/FZFtAN4C8COl7Cm1A81HTApERKay7Ny4UmohfB3I+mWP6B7vBnC5nTH4SaCmkIy9ERGlJ/dc0Sz+PgVmBSIiM65JCv7mI06dTURkznVJgbdoJiIy556koL1TNh8REZlzT1LwNx8xKRARmXJdUuDM2URE5tyTFLR3ypoCEZE59yQFDkklIorIdUmBzUdEROZclBR8/7P5iIjInGuSgohAhM1HREThuCYpAL4mJE6IR0RkzmVJgRPiERGF47KkIOxTICIKw3VJgTmBiMicy5IC79FMRBSOu5JCBpuPiIjCcVdSYPMREVFYLksKbD4iIgrHVUkhk81HRERhuSopiAivUyAiCsNVSSFDeI9mIqJwXJYU2HxERBSOC5OC01EQEaUudyWFDM6SSkQUjruSAmdJJSIKy1VJIVOE1ykQEYXhqqTgu8mO01EQEaUuVyUFrwIqG1qcDoOIKGVlOR1AMh2sbMTBykanwyAiSlmuqikQEVF4TApERBTApEBERAGuTAq8gI2IyJhLk4LTERARpSZXJgVe1UxEZMydSYFXNRMRGbI1KYjIZBHZJyIFIjLVZJ2bRGS3iOwSkX/ZGY8fKwpERMZsu3hNRDIBvAjgWwBKAGwSkflKqd26dUYCeBDA5UqpGhE52a549Nh8RERkzM6awngABUqpQqVUK4C3AVwfss5PAbyolKoBAKVUuY3xBLD5iIjImJ1JYQiAw7rnJdoyvVEARonIGhFZLyKTjTYkIneJSJ6I5FVUVMQdGIekEhEZszMpiMGy0NI4C8BIAF8DcCuAv4lIvy4vUmqWUmqcUmrc4MGDYw7ogUmjALCmQERkxs6kUAJgmO75UABHDdb5SCnVppQ6CGAffEnCFn17dgPAPgUiIjN2JoVNAEaKyAgR6QbgFgDzQ9b5EMDXAUBEBsHXnFRoV0AZWt2lpKbZrl0QEaU125KCUqodwL0AFgPYA+AdpdQuEZkuItdpqy0GUCUiuwF8BuDXSqkqu2IqrPBNm33PnHy7dkFElNZsvZ+CUmohgIUhyx7RPVYA7tf+JU1jS3syd0dElDZcdUVzltZ+xD4FIiJjrkoKIr6k4PU6HAgRUYpyVVLwdzR7WVMgIjLkqqSQyeYjIqKwXJUUMrTmI+YEIiJjrkoK/poCEREZc1VSYEogIgrPXUmBWYGIKCxXJQUiIgqPSYGIiAKYFIiIKIBJgYiIAlyVFM4b0hcAcHrf7g5HQkSUmlyVFL5+zskAgG+ee4rDkRARpSZXJQUAGNSrG+c+IiIy4bqkICJMCkREJlyXFDJF0OFlUiAiMuK+pJAh6OD9FIiIDLkuKWRk8H4KRERmXJcUMtmnQERkynVJIYN9CkREptyXFDIEn+4pQ2VDi9OhEBGlHNclhfYOLzxtXvzwtY1Oh0JElHJclxT8DUfFVY2OxkFElIpclxQyeKcdIiJTrksKByt9NQQOQCIi6spSUhCRX4pIH/F5TUQ2i8gku4OzU0t7h9MhEBGlHKs1hTuUUnUAJgEYDODHAJ60LSoiInKE1aTgb4i/BsDflVLbdMvSEi9VICLqympSyBeRJfAlhcUi0hsAZxAiIjrBZFlc704AYwEUKqWaRGQAfE1IRER0ArFaU5gIYJ9S6riI3AbgIQC19oVFREROsJoUXgbQJCIXAPgNgGIAb9gWFREROcJqUmhXSikA1wN4Vin1LIDe9oVln6lTRjsdAllwy6x1+PW725wOg8h1rCaFehF5EMAPASwQkUwA2faFZZ+MtB4z5R7rC6vxbn6J02EQuY7VpHAzgBb4rlc4BmAIgKdsi8pGnOaCiMicpaSgJYI3AfQVkWsBeJRSEfsURGSyiOwTkQIRmRpmve+KiBKRcZYjjxGTAhGROavTXNwEYCOA7wG4CcAGEfluhNdkAngRwBQAYwDcKiJjDNbrDeAXADZEF3ps2HxERGTO6nUK0wBcopQqBwARGQzgUwDvhXnNeAAFSqlC7TVvw9dRvTtkvScA/AnAA1HEHbMMZgUiIlNW+xQy/AlBU2XhtUMAHNY9L9GWBYjIhQCGKaU+thhH3ITNR0REpqzWFD4RkcUA3tKe3wxgYYTXGJW+gRmHRCQDwEwAP4q0cxG5C8BdADB8+HAL4ZpjRYGIyJzVjuZfA5gF4MsALgAwSyn12wgvKwEwTPd8KICjuue9AZwP4HMRKQIwAcB8o85mpdQspdQ4pdS4wYMHWwnZFDuaiYjMWa0pQCn1PoD3o9j2JgAjRWQEgCMAbgHwfd32agEM8j8Xkc8BPKCUyotiH1FjTYGIyFzYpCAi9dA1+ej/BEAppfqYvVYp1S4i9wJYDCATwGyl1C4RmQ4gTyk1P464Y8aaAhGRubBJQSkV11QWSqmFCOl7UEo9YrLu1+LZl1VMCkRE5lx3j+YM171jIiLrXFdE9sjurBzlF9c4GMmJY+eRWhRVNjodBhElgOuSwqQxpwQery2odDCSE8e1z6/G1/78udNhuNrqA5Uor/c4HQadAFyXFHhFM52IbnttA7778jqnw6ATgOuSgp7RsCqidHWousnpEOgE4OqkQETJVVrbDK+Xp2OpjEmBiJLicHUTJs5YjmeXHXA6FArD1UnhmaX7cazWg9ypC/AAb/1IZKuyOl9H+GoO8Ehprk4KgG84JQC8x1s/EhExKRClO6XYRk+J4/qkwFkviCgZRjy4AI//e5fTYUTk+qRAlO5YUUgPSgF/X1PkdBgRMSkQpTnmBEok1ycFNh8REXVyfVIgSnfsaKZEYlIgIqIA1ycFAduPKL2xnkCJ5PqkQJTu2HpEicSkQEREAUwKRGlOsQGJEohJgSjNsfmIEsn1ScHLXxQRUYDrk0JDS7vTIRARpQxXJoVLcvsHHv/y7a0ORkJElFpcmRTm3Hmp0yEQJQxbQCmRXJkUumdnOh0CUcIkc/RRW4cXI6ctxDt5h5O2T0ouVyYFIopNg6cdbR0Kf1i4x+lQyCZMCkRpjs1HPhP+sAy3z97odBhpz/VJ4YqzBzkdAlFcmBN8jtV5sGJ/hdNhpD3XJ4W2Dq/TIRC5ApNXenB9UthwsNrpECiJ/rG2CLlTF6TM9Sm3z96IUdMWxbUNp+6nsHjXMSzZdSzp+911tBYV9S1J369buD4phCqv8+Do8WanwyCbvLb6IACgqiE1CpUV+yvQmka1VX36uXtOPu6akx9+faVw2YxleC+/JGExfPu51fjG059bWnfLoRq0tHckbN9u4NqksPPxqw2Xj//DMlz25PIkR0PJ9tTifU6HkDCp3CzT7lU4WuvB1Pe3J3S79Z7INb1DVU34zktr8ehHuxK67xOda5NCr5wsp0MgB328vdTpEBImma1H6XRLqtrmNgDAzqO1DkeSXlybFNJRWZ0Ht85aj+NNrU6HkjRKKTyzdD8OVTU5HQohtWsllBhMCmnk1RWFWFdYldD22VR3qLoJzy07gJ+8scnpUFJXCpfUvIYi/TApUErzaoVKa3v6dMYmWzKnuYi2+Yg3AEo/tiYFEZksIvtEpEBEphr8/X4R2S0i20VkmYicYWc8RCyk4hPt0fPXFMSBzgh+1rGxLSmISCaAFwFMATAGwK0iMiZktS0AximlvgzgPQB/siseohMVm2gokeysKYwHUKCUKlRKtQJ4G8D1+hWUUp8ppfw9iOsBDLUxHqK0daCsHkt3lzkdRlqNPpK0ijZ12JkUhgDQz69boi0zcyeA+C7tpBNWok6G07Wg+NbMlfjpG3mGf0tmRSHW5iNKH3YO1jf69Rl+RUTkNgDjAHzV5O93AbgLAIYPH56o+CiNFFc14XhTK/r17OZ0KCnHqWkurGC7fvqxs6ZQAmCY7vlQAEdDVxKRqwBMA3CdUspw7gGl1Cyl1Dil1LjBgwfbEiylviW74m8+YSEVn2gTUArnK1ttKKxCeZ3H6TBiYmdS2ARgpIiMEJFuAG4BMF+/gohcCOBV+BJCuY2xRG1/WT08bZwz5UTjPQFHtqZ085EtUaS+m2etxzXPrXY6jJjYlhSUUu0A7gWwGMAeAO8opXaJyHQRuU5b7SkAvQC8KyJbRWS+yeaSQn+LwUkzV+K+t7c6GA11kZ7dAbZL5bNxf83Cib4cp2uFlbpJF1O5iS+UrRMAKaUWAlgYsuwR3eOr7Nx/JDNvvgC/mrst8Pzx+cETZ204WJXskIhSWqxlm9MFNFnn6iuaT+ndPeh5Y2twc5GX32NKA8kscP37snre7+RPKJVGmqVRRcHdSSES/yyLdOLwRvnrnLOuCP/xfIq3DTtQ4FjdZeCKZhc2H6Urzh9NJzSlFJQCMjJEex7d6x+2aS7+6sZW9OuRbcu2bZWGPc2pUGNIgcNgGWsKdEKb/JdVGPVQal0TWdnQgoueWIqZn+5PyPacGH1kvfnI+eIwUTHMWLQHn6bAVeV2c3VScP7r6qwth2ow6qFFQaMkTjT7yurRruscSoVCyn9/4URcd+GUaJuP9JJ13p7oGsKrKwrxE5OryiNJp9FHrk4KGVFM3djhVfCeYD3Pf11ViNZ2LzYUVjsdijUJOPyp9NtM1MyhyXxP0e4rsHqC3uuts9ZHse8U+rDTiKuTwqUjBmDimQMtrTv28SW48k+f2RwRhUr0WWUqFROJKsxTefSR7oUJsa4wPYeJp9L3LhJXJ4WMDMFbd03A1CmjI65b39KOI8ebkxBV8vGMiqJlvfnIue9WKnQwx2P1gUqsOlCR9P26Oin4pfdXJ3bp8KNJdJHipuaj8joPlu9NbL9FzM1HDkilk51Yvne3vbYBP3xtY+KDiYBJAc7cFYqckjoFRaKYvaPvvrIOd7weW8dopH1ZHn0U7QtskOyTH6PaUSolqEiYFAD851jz2zx42jrwj7VFgecF5Q3YENKuWdnQgjMfXIC8ojTpsE1TFQ0teHDejrju15zsmsLiXccw7YMdjsRwqLpJ21/id2i5+SgFCsNkx5BKtdFYMCkAOLlPd9O/jX74EzyqmxPpqmdW4OZZ61Fa24yth49DKYW8omp4lW80T6ibXlmH2asPdlk+b3OJI+2FRuz+Em88WI2Glva4t/PU4n14a+MhLN51LOZtxPtWj9V6sO3w8aBl1Y2tyJ26AB9tPdJl/bvn5OPNDYfi3Gt4kQr9eD/feZtLcDjWBJPA71a6DOs0ijJNQgfAK5pjNnHGcgDAE9efF0gqRiNWNxZVY2NRNe64YkTQ8vvf8U3EV/Tkt+0N1GFVDS246dV1+Obok/Hajy6J+vXGVfHEmbvpELpnZ+L6MLVFva8+9Rla2r1Bn9sXFQ0AgDfWFVvaTtLPXON5rVK4/51tGHhSN2RnZuD7l/pucuVEa1CsI8JTofkonTApxOlAeQNO0ZJCun4Z7OxTadbuSbH3WH1g2etrDmJPaT3++N0vR3x94juag7f42/d9TTtWk0JLHE1Xdon0tfO959g+ZP+2qxpbAQDPLI3uKuxEfn6x/r5SoQkrnbD5KE4CQCS2eXVSRbLjfuzfuzE373DkFZH42Mw2t0+XtOxm15mrWXKPq6YQx2uBxH5+0W7KqdF1aVoMBDApxElEoM21FvUMnI5Li1FXBs1HcRxns5f+JcI8RImsBQYuAEvSsLfQ7+X+snrL72dPaV1c+07kWXr03RnO/B6N4kynooFJIU55xdWB6TKsfO6FFQ3IL66xNyiXOt7UivaO8M07MTdBpPCPOnLzUefj/OIaTJq5Ei8sL8Dxptawr6tubMW1cU4b3jl1dvxiLeST3qeQ5nUFJoU47TxSF6i2exWwvrAK172w2nTY5DeeXoEbX16bxAjTm9XCuK3Di7HTl2LaBzvDby/WOCysE23Rk6iiKppCqKTGN4ro6aX7MXb60rDrNifgHuWJ7VOINYbEFtJvbQw/msywpmAxhv9bsBs3vLQmlrAShklB87OvnxXzazv7FBR+N28HtpfU4rD240sHqXxeYzW2Nq2GMH/b0cRsMPRlMZRI1Y3GZ+KRNrXlUA08MRTISsHwZETffBTN28hMQPNWtMfteFOr6ZDjVKmtPThvR+SVNB9uOYLrX7ReyP911UFsPnQ88oo2YlLQ/Prq0Xj+1gtjem2VNvW0UqldwIYK/ckXlDcYXk9Q72kLnGEmWluHFw/O2246r1S0BUFrhxe5UxcEXXCYCLF8rhc9Ef5MXK9GSyCltc34zktr8bsoCh79MTK6VibWwjTDgdLhv/+5GXfPyUd5nafL39Km+UgX5n1zt2Lb4eNByx79KHxt1mlMCjqXnWVtxtRQU7UfsF0dzTOX7sfFURQwsbrqmRW4fXbXuVaue2ENrvhjfDPEHjnejAl/WNYl6awuqMRbGw+bFoLRFgQd2mD255cfMNmesWja5QEE9V0k4mP3z9Nf7/Ednx1Hak3XPd7Uimbd/cT1uze6haz+73b3bX+45QheWfFF576jPDb+q7BbDfqGUqWmEEmk7+w/1hUbLm9pj7+5LhGYFHQG9srBL745MurX+avs7Tbdb+HZZQcC48TtZtQJfrCyEYDvat72Di/aO7x4Zsk+1DZFvoe1/od8rM6DfcdCRrNEWRgDwPK95V2aJbquZ1z6WWnOKChvwGd7y4NfFxLo2dMWJfSKdP8xtlLwjZ2+FOc+8glufHktlFIRT0aS2bl+39yteHLRXlQ1tCC/uAY1us5sK9sLF2syc0J5vQejpi3qcvW6FUZvYcGO0oivu+P1TVHvyw5MCiEmnDkg5tc2eDrPgpVSQWfF0/+9GzOjvPAnWh9vP4rcqQtQWhv9FN9WCo4JM5bh9wv24LN9FXhueQEe/3f09y+2Mgwz0giij7YexbzNwVNKWK2lmdYUdH+56pkV+HHID9Ro8yv2BSeF481t+M1724LO4sMxOhSdw1Ujvz6/uAbtXhX02Rm9zMq5iqetA2sKKkNeF3sxXNPUhhtfXhvVTXEA3YR7BgcgkcOCS2qa8Pm+csO/7TxSi5/8Iw+tHV68HkMzpFGUv3lve8TXrSlIjXtFMCmEuOysQdj7xOSYXrtbN6b728+txmurOuc8mr3mIJ5dZtykEarDq3D3nDxsOWRt6Opzyw7gp2/k4d28EgDBVw8b+WTnMXzj6c+DfvQdFms5y/aWIVP71sRSewm9251RIbjzaOdxNCsHjoW0OVutpNnZBFFQ3oB38kowd1P8cx1ZbQf3KhX83g1e9vSSfRFvufrQhzvxg79tQEF5Q2BZIo5Vo5YgRaIr1GNNbnrhdnf1zJX40d+7npkfKKvHtc+vxvYS8+a7yPtNk3YuE0wKBrpnZ2LuXRNieq1/FExLu9fyjdkbWtpx/ztbUdXQgkU7SvH1P3+OxbvKcO+/thiuf6zWg3+u72yXfGbpfizdXYaVWnNGpC/lb9/fjsKKxkD7NWD+gwsdBaMU0CPbNzuK0Rnx/G1Hg0aP7Doa/OPKMCnr9It7ZGd27s9qo0HIalbOtO+eEzyt9IyFe/Bvk9FLxsMMrdN/Jonse1Iq8vbeWFeMcb//FLlTF5gOlT5Q5juRqPd0NgnGF6fBRYdxbC2eDRh9hxpNanO3/nVD0HP/1yiagj69UwLnPjJ1qcXbdIYqqYm+6eb8RxcDAJpaOvCJrkCtamwxHIXxN23W1UnnnYKTe3fO8Or/3np1v/s564vROycLFw7vh7rmdnxpaN9Agelf36xdurzOg/F/WNZluX+kUI3BxWK/eMuXyDY//C1kZgju+efmoL93qSkY/IL0naW//3hP1xUMhMZvlhP0BcTiXcE3oHl1ZdeRO/rtm9UArBQYs1YWYsr5p2H4wJ5xnYGH7surVNDnHUlThKatLyoaceHw/tq+og4vIEILYEyiHXQQS/jtIQezqbUDhRUNyB14kvX9pnlWYE3BYfozt8bW4JE5njavYaHsZ9bkoy8gH/5wJ+6buxVffepz/McLvqtTjavmna8prW1GcVUjSmsNhgUq4IF3fTO8HihvwP9qj0Nd9MRSXDh9SZfloUnBT9+GfNOr6wKPrd6Tt0tSMJsHKOazzc7J82IxY9Fe/NfsDahuNB+HD+iuALYYf4c3ckeznmkNSvvDA7rPM56agtFrrXU0a//DdyFoWwyjvKZ9sAOeto5AAo1mSGroPj7ZdQzfeHqF5RFcO4/UBoaoJ1Kdp820FptoTAoOG/XQosDjSDNwhp4pmZ1FR2p79RfA/h/ue/klQQlm4ozl+OpTnyMrM/KP6aOtnV/U0CF1RnGEDhWN56xq9uqD2KuNZrK6mViHpH7H4AKkcEfHqPZQXN2Ei55Yipc+9w3ZDC1cthyqwTGDRBy03ZDnh6ubg2LfGWYoa/gNd403nsF0sSYU/3d8c3ENbpm1PmhwhtUtvrnhED7YciRo/R++tgEPfRh7UjdLTq3tXvzsX5sDo8eufX41rnluVcz7MfPrd7fh529tCTTz2YlJIYW0RZy3J/i52TA3pRQ2H6oxHQXjL4v8V9yuOlBpWAAs39N1dEZoYdc9u/MrdM5Dn5hE3mnRTuOz5FiGz0//eDeuedb3A4y3nd5ofL/eAV0HrN76wir8zKDvx+h4hgvR09aB77y0NjDqyauU4bj1DQeDa07XPLcKq3WjhhI5giWeYxpuqoe84hrkTl2A/WEKuDKt2TS44zuKdv2QVVcdqMQ/14cfAFBY0WD6PWhpM/5tbiqqxoLtpUF31/OYrKtX29yG0Q8v6jLiy4y/WdrKtuPFpBBGn+7J7XKJNJTR6giho7Ue3PDSWvzuA+MzI/+oIf0oJa/Btp82GEIbulZ7R+wFR2ltMx760Hd157K95dgRxYgP/2gaf9ihhYBpk4FJuBsOxnYr1Ttf39RlZI+CtQK1uKrzKvHQtv79ZQ2GSbb0eNeaxB8/2WsxWuPP2Uw8edbK+1/3hXkCa9O+V5m6kQnR1FyW7j5m6b3qE82UZ83P8M0uLCsz6POzYueRWnjavKYjEnOnLohpu4nApBDGmqnfSOr+Ig0ltXrm5p/9cvfRrtMe/6fJPCwei1dTdrmyN442hokzlgcNLf3l28ajrYz8fU1R0HPr1ykkthewR7euJw6z1xzscg2DEf3oL6usnhiYsfLyRu36mniGVobGKZAuCVl/m9tQ/gEMGbqkEM1n99m+CszRjdAzow8zXPNt6N+uf2E1GlvaA3dQjJY30N+RepgUwujdPRv9e2YDAD69/6sORwPL9/r1d14b9QlsNblC03970UjsnBY4rgGQFoektsVRszHSo1vXn9Dh6ubAtBWJFu9V81aS59V/WamtG/t+QpOCle9Ne4cXZXW+Wleb9vq65jY0trRjxqI9pk04re1elNd3PWM/ajKflp7Vk4nQpLCtpDZoYEi004foj09HyAWIoc59+JOgWqXdmBQiuHSEb2jq4N45QcvfvWeiE+F0Mf3fu7ss83+BdxnUFOLl/9HawdPWEThLjZYd806FXmNhJMvmWeM+3V0WNOw3dMhktEIPU35xNdo6vEFFtr/92uox/d93tuHLjy0OWhaavARimhaOHm/GB1tKgmaV9b/nVQcqcd6ji/HqikJc+afPDF//q3e2Yvz/dR2lZxb+Xt1UK16lUFzVGLHpMtK8RJ42b8x9Hmf9biEeC1Nram7rMJyo0i68TiGCmTePxc8rG9C3Rzbm3jUBawoqcemZA3FJbud0GK/cdjHu+We+I/HNXnOwyzKjycTSQWmtB+c9ujjyigZCf4+t7d7A9R+xijRFQ4dSgVEndvnJG3n41VWjAs+t9OFsKqoO+n7qdYQcqBtfXoc7Lh9huK7VpPD+5pKu+wlJCuFGsn3/r+tRVNWEVb/5uunrw1mwPfK8Qn7l9R5M/ktn38HVM1eiyMJZuNFFf/rDU17viapfyl9z8r/GbJI8J7CmEEGPbpk47/S+AHwXtN0/6RxcfvYgAMCZg3wXtEw+/1RsfeRbjsUYyuyq1RPVC8sPdEkKtc1tQWdXay2O8tCL1FQT2q9hF/29OawUlt97ZR3mGRTUAPAkbE9XAAAO2klEQVTkoq6d0juOdG1SrGlsDRr5E63QY5edmWE63Paotlw/+s5qM1+0Z9Ch/ThWEgJg3N8QOlowXMd5qFQ+b2NNIQ4f3nt5YBK8fj27BZZvmnYVJs1cgRoLs4jaYbPFOZNOFH9esh/VjcHHOrRI+f7fNuCD/7ksqu3aNetttPSjaKzGFE0HaKtBATzl2VVd5peKRkdIM1d1Y6vphY5++uGWVprJpr6/HW9vOmz6d6MjFWsro1FS0NfaDlc3W57bDIh/wICdWFOIQ5/u2Ti9X4/A85En9wIADDipGx6+doxTYaGwwt4mjVQU2oxm9KP7zkvR3QY1VWpc87Z0zghbWBH72bsZo1lpIyWESCc8b280L6z19AlPf/tPKzWFcAkB6JwGPvjeFLEVxo8btPmX18fevxbzdOZJmFnJ1qQgIpNFZJ+IFIjIVIO/54jIXO3vG0Qk18547Lb4vq9g9/SrkZkhuOGiobhypK+Z6b6rRuKNO8Zb2saDU0bbGSKluXfzjZuF4rHraF1cs4L63fuvznmuluwuC7Nmp6KqxkDy1d+7PNL06bEKne/KqkKDviP9dCzRKq6ObTRRMmoYtiUFEckE8CKAKQDGALhVREJPn+8EUKOUOhvATAB/tCueZMjIEPTUjVv/y81j8dvJo/HLb47EV0YNxohBwZNqffzzK7Dol1cGnr97z0Tc/dXOe0W/cttFAICBJ3XDpDGnhN336FN744KhfRPxNohi8nEUHb5+33h6heHyeJquwlm2J7akkGhGfTtWVMRRO7FK7Jr7W0QmAnhMKXW19vxBAFBKzdCts1hbZ52IZAE4BmCwChPUuHHjVF6ePWPAk2V9YRVO79sDwwf2BOC7srdvj+yghOJX3diKXjlZKKxswOS/rEL/ntl4+NoxOLVPd3z/bxuw5eFvoaapFaf06Y6TcrLw4LwdeGuj8fUM/Xtmx93PMe9/LsMNWjPMyb1z4qpCE1F0rrvgdDwX473kRSRfKTUu0np2djQPAaBv9CsBcKnZOkqpdhGpBTAQQPRDRdLIhJBpuU/r28NkTV//BACMPrUP9kyfjB7dOu81UPTktwEA/U/q7OSeccOXcMXZg7BwRykevW4MMkQwqJfvGgulFFravVixvwITRgzEu/mHUdHQgj7ds7F0dxl65WRh4lkDcd0Fp6O01oNhA3rg8fm7cePFQzF2WD9kZggGnNQNC35xBU7t0x0De+UELsc/97Q++PHluYE7TD04ZTRmaGdDD0wahT8vsX7XucX3fQWf7DyGGy8egurGVlz3gvFV2LF6+nsXBHV66ocU/+iy3JjuthWrnt0yu0xxccbAnkm9WInSx/ABPW3fh501he8BuFop9RPt+Q8BjFdK/Vy3zi5tnRLt+RfaOlUh27oLwF0AMHz48IuLi1NnTC+Z21FSi4G9uuH0fj2wYn8FJp45EN2yglssCysaMKR/DwgEDS3tOCknEzlZmUHr+DsjW9q96FAKe0rrMHZYP6wuqMS4M/ojKyMDZXUe7C6tw6QxpyArs3MfdZ42lB734OTeOTje3BZowqttbkNRZSO+PLQvRAQV9S1oaGkP/H3r4ePo2yMbWw/X4PzT+6K2uQ1jh/VDY0sHOpTC0ePNOK1vd7y2+iC+pTXt+e9DsPXwcTS1tqO13YsvDemLD7cexaUjBmB7SS0uye2PI8ebUVDegDuvGAERgVIKNU1tyMoU9M7JgohgQ2EVRp/aB19UNqC13Yt+PbOx60gdbrhoCEpqmvHx9lL0yslETnYmvn7OyejXMxvPLy9AeZ0HpbUe/ODS4eiVk4WszAzUNrehZ7dMZGdmIDNDsO6LSpw5uBd6d8/CKX26Y0dJLY43t/liq2lGcXUTTu3THTVNrRg+oCeeX16AByadg6xMwfrCKtw24QxkimB1QSUG985BhghKa5uRnZmBD7YcwZjT+uD8IX3R4fWipKYZ55zaG20dXpzcuzsOVjbigy1HMGnMKfjS0L5YvqccV44ajJX7K/D88gOYOuVceNo60KdHNk7unYPhA3rivfwS1DS1Yt7mI5hz53jUNrdh2+HjKKpqwrFaDyaeNRDFVY0Y1CsHZwzsidP79cC+Y/XYUFiNy88ehLJ6D8YO7YfX1xah/0nZuOHCoXh/cwmO1Xlw5dmDcN7pfTE37zB2HqnFr68+BzuO1OKLigYcqmrCQ9eOQXmdB5UNrXh9bRHGDuuHivoW9O6ehe7ZmRhzeh/kZGVgxKCTsHxvOUaf2gcXDu+HtQWVEBHUNrfh8rMHYUNhFZraOlDd0IovD+uLV1cUYnzuAAwb0BMVDS1obe9AfnFNUAf7VeeegovP6I9P95Sh3tOGlnYvFt/3FXTPDv59WGW1psDmIyIiF7CaFOwcfbQJwEgRGSEi3QDcAmB+yDrzAdyuPf4ugOXhEgIREdnLtj4FrY/gXgCLAWQCmK2U2iUi0wHkKaXmA3gNwBwRKQBQDV/iICIih9h6RbNSaiGAhSHLHtE99gD4np0xEBGRdbyimYiIApgUiIgogEmBiIgCmBSIiCiASYGIiAJsu3jNLiJSASDWS5oHITWn0GBc0UnVuIDUjY1xRedEjOsMpdTgSCulXVKIh4jkWbmiL9kYV3RSNS4gdWNjXNFxc1xsPiIiogAmBSIiCnBbUpjldAAmGFd0UjUuIHVjY1zRcW1crupTICKi8NxWUyAiojBckxREZLKI7BORAhGZmuR9DxORz0Rkj4jsEpFfassfE5EjIrJV+3eN7jUParHuE5GrbYytSER2aPvP05YNEJGlInJA+7+/tlxE5Dktru0icpFNMZ2jOyZbRaRORO5z4niJyGwRKReRnbplUR8fEbldW/+AiNxutK8ExPWUiOzV9v2BiPTTlueKSLPuuL2ie83F2udfoMUuNsQV9eeW6N+rSVxzdTEVichWbXkyj5dZ2eDcd0wpdcL/g2/q7i8AnAmgG4BtAMYkcf+nAbhIe9wbwH4AYwA8BuABg/XHaDHmABihxZ5pU2xFAAaFLPsTgKna46kA/qg9vgbAIgACYAKADUn67I4BOMOJ4wXgKwAuArAz1uMDYACAQu3//trj/jbENQlAlvb4j7q4cvXrhWxnI4CJWsyLAEyxIa6oPjc7fq9GcYX8/WkAjzhwvMzKBse+Y26pKYwHUKCUKlRKtQJ4G8D1ydq5UqpUKbVZe1wPYA9896c2cz2At5VSLUqpgwAK4HsPyXI9gH9oj/8B4D91y99QPusB9BOR02yO5ZsAvlBKhbtg0bbjpZRaCd+9PkL3F83xuRrAUqVUtVKqBsBSAJMTHZdSaolSql17uh7A0HDb0GLro5Rap3wlyxu695KwuMIw+9wS/nsNF5d2tn8TgLfCbcOm42VWNjj2HXNLUhgC4LDueQnCF8q2EZFcABcC2KAtulerBs72VxGR3HgVgCUiki++e2EDwClKqVLA96UFcLIDcfndguAfq9PHC4j++Dhx3O6A74zSb4SIbBGRFSJypbZsiBZLMuKK5nNL9vG6EkCZUuqAblnSj1dI2eDYd8wtScGo3S/pw65EpBeA9wHcp5SqA/AygLMAjAVQCl8VFkhuvJcrpS4CMAXAz0TkK2HWTepxFN9tXK8D8K62KBWOVzhmcST7uE0D0A7gTW1RKYDhSqkLAdwP4F8i0ieJcUX7uSX787wVwSceST9eBmWD6aomMSQsNrckhRIAw3TPhwI4mswARCQbvg/9TaXUPABQSpUppTqUUl4Af0Vnk0fS4lVKHdX+LwfwgRZDmb9ZSPu/PNlxaaYA2KyUKtNidPx4aaI9PkmLT+tgvBbAD7QmDmjNM1Xa43z42utHaXHpm5hsiSuGzy2ZxysLwA0A5uriTerxMiob4OB3zC1JYROAkSIyQjv7vAXA/GTtXGuzfA3AHqXUM7rl+vb47wDwj4yYD+AWEckRkREARsLXwZXouE4Skd7+x/B1VO7U9u8fvXA7gI90cf2XNgJiAoBafxXXJkFncE4fL51oj89iAJNEpL/WdDJJW5ZQIjIZwG8BXKeUatItHywimdrjM+E7PoVabPUiMkH7jv6X7r0kMq5oP7dk/l6vArBXKRVoFkrm8TIrG+DkdyyenvN0+gdfr/1++LL+tCTv+wr4qnLbAWzV/l0DYA6AHdry+QBO071mmhbrPsQ5wiFMXGfCN7JjG4Bd/uMCYCCAZQAOaP8P0JYLgBe1uHYAGGfjMesJoApAX92ypB8v+JJSKYA2+M7G7ozl+MDXxl+g/fuxTXEVwNeu7P+OvaKte6P2+W4DsBnAf+i2Mw6+QvoLAC9Au6A1wXFF/bkl+vdqFJe2/HUA94Ssm8zjZVY2OPYd4xXNREQU4JbmIyIisoBJgYiIApgUiIgogEmBiIgCmBSIiCiASYFcS0TWav/nisj3E7zt3xntiyjVcUgquZ6IfA2+WTyvjeI1mUqpjjB/b1BK9UpEfETJxJoCuZaINGgPnwRwpfjmzv+ViGSK794Em7RJ3O7W1v+a+Oa+/xd8Fw5BRD7UJhPc5Z9QUESeBNBD296b+n1pV6I+JSI7xTcv/826bX8uIu+J754Ib2pXuxIlVZbTARClgKnQ1RS0wr1WKXWJiOQAWCMiS7R1xwM4X/mmegaAO5RS1SLSA8AmEXlfKTVVRO5VSo012NcN8E0MdwGAQdprVmp/uxDAefDNWbMGwOUAVif+7RKZY02BqKtJ8M0vsxW+aYwHwjf/DQBs1CUEAPiFiGyD7/4Fw3TrmbkCwFvKN0FcGYAVAC7RbbtE+SaO2wrfzV6Ikoo1BaKuBMDPlVJBE4ppfQ+NIc+vAjBRKdUkIp8D6G5h22ZadI87wN8nOYA1BSKgHr5bIfotBvDf2pTGEJFR2iyyofoCqNESwmj4bo/o1+Z/fYiVAG7W+i0Gw3ebSDtndCWKCs9EiHwzVLZrzUCvA3gWvqabzVpnbwWMb7v4CYB7RGQ7fLN8rtf9bRaA7SKyWSn1A93yD+C7x+82+GbH/I1S6piWVIgcxyGpREQUwOYjIiIKYFIgIqIAJgUiIgpgUiAiogAmBSIiCmBSICKiACYFIiIKYFIgIqKA/wfNmO4d0+AdQAAAAABJRU5ErkJggg==\n"
                    },
                    "metadata": {
                        "needs_background": "light"
                    }
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2 id=\"Question_3\">Question 3:Find the misclassified samples</h2> "
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<b>Identify the first four misclassified samples using the validation data:</b>"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "sampleSeq=0\nN_samples=0\n\nfor x_test, y_test in validation_loader:\n    model.eval()\n    z=model(x_test)\n    _, yhat=torch.max(z.data,1)\n    \n    for i in range(len(y_test)):\n        sampleSeq += 1\n        if yhat[i] != y_test[i]:\n            print(\"sample#:{} - predicted value:{} - actual value:{}\".format(sampleSeq, yhat[i], y_test[i]))\n            N_samples += 1\n            if N_samples >= 4:\n                break\n    if N_samples >=4:\n        break",
            "execution_count": 36,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "sample#:9 - predicted value:0 - actual value:1\nsample#:320 - predicted value:1 - actual value:0\nsample#:493 - predicted value:0 - actual value:1\nsample#:694 - predicted value:1 - actual value:0\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<a href=\"https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/share-notebooks.html\"> CLICK HERE </a> Click here to see how to share your notebook."
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2>About the Authors:</h2> \n\n<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD."
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Copyright &copy; 2018 <a href=\"cognitiveclass.ai?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu\">cognitiveclass.ai</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>."
        }
    ],
    "metadata": {
        "kernelspec": {
            "name": "python3",
            "display_name": "Python 3.6",
            "language": "python"
        },
        "language_info": {
            "name": "python",
            "version": "3.6.9",
            "mimetype": "text/x-python",
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "pygments_lexer": "ipython3",
            "nbconvert_exporter": "python",
            "file_extension": ".py"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 2
}